Masked Face Recognition Using Multi-Task Cascaded Convolutional Networks and FaceNet

Face masks have been widely used since the start of the COVID-19 pandemic. Facial detection and recognition technologies, such as the iPhone's Face ID, heavily rely on seeing the facial features that are now obscured due to wearing a face mask. Currently, the only way to utilize Face ID with a mask on is by having an Apple Watch as well. As such, this paper intends to find initial means of a reliable personal facial recognition system while the user is wearing a face mask without having the need for an Apple Watch. This may also be applicable to other security systems or measures. Through the use of Multi-Task Cascaded Convolutional Networks or MTCNN, a type of neural network which identifies faces and facial landmarks, and FaceNet, a deep neural network utilized for deriving features from a picture of a face, the masked face of the user could be identified and more importantly be recognized. Utilizing MTCNN, detecting the masked faces and automatically cropping them from the raw images are done. The learning phase then takes place wherein the exposed facial features are given emphasis while the masks themselves are excluded as a factor in recognition. Data in the form of images are acquired from taking multiple pictures of a certain individual's face as well as from repositories online for other people's faces. Images used are taken in various settings or modes such as different lighting levels, facial angles, head angles, colors and designs of face masks, and the presence or absence of glasses. The goal is to recognize whether it is the certain individual or not in the image. The training accuracy is 99.966% while the test accuracy is 99.921%. © 2022 IEEE.

Baseline Estimation in Face Detection for AI Proctored Examinations through Convoluted Neural Networks

The COVID-19 pandemic took the world by surprise, and everything came to a halt. The education sector had to adjust accordingly by shifting to online learning. If the online delivery experience was overall successful, assessment integrity becomes questionable as examinees still manage to circumvent the anti-plagiarism mechanism put in place. In this paper, we propose an artificial intelligence solution using face and head pose detection to estimate the neutral position of the examinee which will form the basis to detect any suspicious behavior. The resulting implementation achieved a 97% accuracy when detecting the examinee in the frame and a 98% accuracy when there are multiple faces detected. © 2023 IEEE.

Covid-19 Control: Face Mask Detection Using Deep Learning for Balanced and Unbalanced Dataset

Facemask wearing is becoming a norm in our daily lives to curb the spread of Covid-19. Ensuring facemasks are worn correctly is a topic of concern worldwide. It could go beyond manual human control and enforcement, leading to the spread of this deadly virus and many cases globally. The main aim of wearing a facemask is to curtail the spread of the covid-19 virus, but the biggest concern of most deep learning research is about who is wearing the mask or not, and not who is incorrectly wearing the facemask while the main objective of mask wearing is to prevent the spread of the covid-19 virus. This paper compares three state-of-the- art object detection approaches: Haarcascade, Multi-task Cascaded Convolutional Networks (MTCNN), and You Only Look Once version 4 (YOLOv4) to classify who is wearing a mask, who is not wearing a mask, and most importantly, who is incorrectly wearing the mask in a real-time video stream using FPS as a benchmark to select the best model. Yolov4 got about 40 Frame Per Seconds (FPS), outperforming Haarcascade with 16 and MTCNN with 1.4. YOLOv4 was later used to compare the two datasets using Intersection over Union (IoU) and mean Average Precision (mAP) as a comparative measure;dataset2 (balanced dataset) performed better than dataset1 (unbalanced dataset). Yolov4 model on dataset2 mapped and detected images of masks worn incorrectly with one correct class label rather than giving them two label classes with uncertainty in dataset1, this work shows the advantage of having a balanced dataset for accuracy. This work would help decrease human interference in enforcing the COVID-19 face mask rules and create awareness for people who do not comply with the facemask policy of wearing it correctly. Hence, significantly reducing the spread of COVID-19.

ViDepBot: Assist People to Tackle Depression Due to COVID Using AI Techniques

Taking care of one's mental health properly is very important as we are trying to get past the effects caused by the COVID pandemic era, especially since the rate of COVID spread is still persistent. Many organizations, universities, and schools are continuing an online mode of learning or working from home situation to tackle the spreading of the coronavirus. Due to these situations, the user could be using electronic gadgets like laptops for long hours, often without breaks in between. This has eventually affected their mental health. The ‘ViDepBot', Video-Depression-Bot aims in helping the user to maintain their mental health by detecting their depression level early, and taking appropriate actions by faculty/counselors, parents, and friends to help them to come back to normalcy and maintaining a strong mental life. In this work, a system is proposed to determine the depression level from both the facial emotions and chat texts by the user. The FER2013 dataset is trained using deep learning architecture VGG-16 base model with additional layers which acquired an accuracy of around 87% for classifying the live face emotions. Since people tend to post their feelings and thoughts (when feeling down, depressed, or even happy) on social media such as Twitter, the sentiment140 twitter dataset was taken and trained using the machine learning algorithm Bayes theorem which acquired an accuracy of around 80% for classifying the user input texts. The user is monitored through a webcam and the emotions are recognized live. The ViDepBot regularly chats with the user and takes feedback on the mental condition of the user by analyzing the chat texts received. The emotions and chat texts help to find the depression level of the user. After determining the depression level, the ViDepBot framework provides ideal recommendations to improve the user's mood. This ViDepBot can be further developed to keep track of each student/subject person's depression level, where they would be physically present in the classrooms, once the pandemic situation subsides. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Face Mask Detection Using Multi-Task Cascaded Convolutional Neural Networks

Detecting faces is a prevalent and substantial technology in current ages. It became interesting with the use of diverse masks and facial variations. The proposed method concentrates on detecting the facial regions in the digital images from real world which contains noisy, occluded faces and finally classification of images. Multi-task cascaded convolutional neural network (MTCNN)—a hybrid model with deep learning and machine learning to facial region detection is proposed. MTCNN has been applied on face detection dataset with mask and without mask images to perform real-time face detection and to build a face mask detector with OpenCV, convolutional neural networks, TensorFlow and Keras. The proposed system can be used as an application in the recent COVID-19 pandemic situations for detecting a person wears mask or not in controlling the spread of COVID-19. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Comparative Study of Various Lightweight Mask Recognition Algorithms

As an important line of defense against novel coronavirus, masks can effectively reduce the risk of novel coronavirus infection. In this paper, three algorithms were used for mask wear detection, respectively using the opencv native library, MTCNN+MobileNet, and pyramidbox_lite_mobile_mask in paddlehub. Finally, the test results of the three algorithms were analyzed and compared, and the experimental results are that the pyramidbox_lite_mobile_mask model in paddlehub has the most sensitive face recognition and mask detection ability, which can identify the blurred face and judge whether to wear a mask, followed by MTCNN + MobileNet. © 2022 IEEE.

REAL-TIME FACE DETECTORS AND CLASSIFICATION PERFORMANCE IMPLEMENTING RESNET-50 MODEL FOR FACE MASK-WEARING CONDITIONS

The World Health Organization requires the community to wear a face mask to avoid transmission of COVID-19. The study investigates the performance of face detectors and evaluates the classification performance based on face mask-wearing conditions. The study built a total of 13,806 datasets that recorded an overall classification performance of 98%. The findings show that Multi-task Cascade Convolutional Neural Networks outperformed the other face detectors with an average score of 70% in accordance to distance, angles, occlusions, and multiple detections across given set conditions. Furthermore, the model recorded an accuracy performance of 83% for "correct wearing of face mask", 91% for "incorrect wearing of face mask", and 95% for "no face mask". However, despite the promising performance rates, the identified best face detector decreases when the given conditions are set to a higher level. To further improve and optimize the face mask-wearing conditions, the study highly recommends employing both statistical and mathematical analysis. © School of Engineering, Taylor's University.

A novel design of a smart interactive guiding robot for busy airports

The COVID-19 pandemic has had a massive impact on the global aviation industry. As a result, the airline industry has been forced to embrace new technologies and procedures in order to provide a more secure and bio-safe travel. Currently, the role of smart technology in airport systems has expanded significantly as a result of the contemporary Industry 4.0 context. The article presents a novel construction of an intelligent mobile robot system to guide passengers to take the plane at the departure terminals at busy airports. The robot provides instructions to the customer through the interaction between the robot and the customer utilizing voice communications. The usage of the Google Cloud Speech-to-Text API combined with technical machine learning to analyze and understand the customer's requirements are deployed. In addition, we use a face detection technique based on Multi-task Cascaded Convolutional Networks (MTCNN) to predict the distance between the robot and passengers to perform the function. The robot can guide passengers to desired areas in the terminal. The results and evaluation of the implementation process are also mentioned in the article and show promise.

Mask Detection for Covid-19 under Extreme Conditions

Mask detection has become a hot topic since the COVID-19 pandemic began in recent years. However, most scholars only focus on the speed and accuracy of detection, and fail to pay attention to the fact that mask detection is not suitable for people living under extreme conditions due to the degraded image quality. In this work, a denoising convolutional auto-encoder, a multitask cascaded convolutional networks (MTCNN) and a MobileNet were used to solve the problem of mask detection for COVID-19 under extreme environments. First of all, a network based on AlexNet is designed for the auto-encoder. This study found that the two-layer max pooling layers in AlexNet could not accurately extract image features but damage the quality of restored image. Therefore, they were deleted, and other parameters such as channel number were also modified to fit the new net, and finally trained using cosine distance. In addition, for MTCNN, this study changed the output condition of ONet from thresholding to maximum return, and lowered the thresholds of PNet and RNet to solve the problem that faces might not be found in low-quality images with mask and other covers. Furthermore, MobileNet was trained using categorical cross entropy loss function with adam optimizer. In the end, the accuracy of system for the photos captured under extreme conditions enhance from 50 % to 85% in test images. © 2022 IEEE.

Anomaly Detection in Red Reflex Images Using Deep Learning Approaches

Amblyopia is a noteworthy disease in children leading to visual loss. This work focuses on creating a deep learning model for the detection of Amblyopia factors in patients wearing masks under the COVID-19 pandemic. © 2022 IEEE.

Tracking Student Attendance in Virtual Classes Based on MTCNN and FaceNet

All classes are held online in order to ensure safety during the COVID pandemic. Unlike onsite classes, it is difficult for us to determine the full participation of students in the class, as well as to detect strangers entering the classroom. Therefore, We propose a student monitoring system based on facial recognition approaches. Classical models in face recognition are reviewed and tested to select the appropriate model. Specifically, we design the system with models such as MTCNN, FaceNet, and propose measures to identify people in the database. The results show that the system takes an average of 30 s for learning and 2 s for identifying a new face, respectively. Experiments also indicate that the ability to recognize faces achieves high results in normal lighting conditions. Unrecognized cases mostly fall into too dark light conditions. The important point is that the system was less likely to misrecognize objects in most of our tests. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Depression Level Determination Using Deep Learning to Help Students in the COVID-19 Pandemic Situation

The Coronavirus disease has affected the mental stability of the students since their academic learning has become completely online due to the stay-at-home order implemented over various states. In this work, a system that incorporates the determination of depression level from the facial emotions expressed by a student is proposed where he/she could be working in front of electronic gadgets like laptops for long hours, due to the lockdown situation. The FER2013 data is used to train the deep learning architecture, visual geometry group model with 16 layers (VGG-16) base model with some additional layers. The model has been used to classify the emotions and has acquired an accuracy of 87.76% on the FER2013 dataset. The emotions are then recognized live, monitoring the student through a Webcam. The multi-task cascade convolutional neural network (MTCNN) architecture has been used for detecting the face live. The depression level of the student is determined by calculating the depression coefficient. The dominant emotions in a depression state, the negative ones were captured quickly which helped in determining the depression level. Appropriate remedies are then suggested according to the depression level detected, to improve the student’s mood and also to maintain their mental stability. The calculation of the corresponding depression level in the student will help the faculty, counselor, parents, and friends to take necessary actions to bring the student back to his or her normal mental stage. The system could become more efficient when the activities of the student could be monitored and incorporated into the current system. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Edge-Oriented Social Distance Monitoring System Based on MTCNN

Social distance monitoring (SDM) systems are vital in fighting the spread of the coronavirus (COVID-19). Existing SDM systems employ bounding box-method, which imposes inaccurate distance estimation due to the high variance in its output coordinates. To solve this problem, an SDM system based on multitask cascaded convolutional neural networks (MTCNN) is proposed. Instead of using bounding box coordinates, face detection and facial landmarks localization of MTCNN is used to provide fixed coordinates and increase the distance estimation accuracy of SDM. However, while the accuracy issue is solved by using MTCNN, the SDM system suffer from large computational requirements due to the cascaded networks added on top of the distance estimation process. To deal with this challenge, a constrained optimization technique is employed to each stage of MTCNN with the goal of reducing its hardware requirements while keeping the same reliability as the original implementation. Experimental results show that the SDM system based on the optimized MTCNN achieves higher accuracy performance with reduced computational requirements as compared to conventional SDM systems. This allows the proposed SDM system using optimized MTCNN to be deployed efficiently on edge devices. IEEE

A novel design of a smart interactive guiding robot for busy airports

The COVID-19 pandemic has had a massive impact on the global aviation industry. As a result, the airline industry has been forced to embrace new technologies and procedures in order to provide a more secure and bio-safe travel. Currently, the role of smart technology in airport systems has expanded significantly as a result of the contemporary Industry 4.0 context. The article presents a novel construction of an intelligent mobile robot system to guide passengers to take the plane at the departure terminals at busy airports. The robot provides instructions to the customer through the interaction between the robot and the customer utilizing voice communications. The usage of the Google Cloud Speech-to-Text API combined with technical machine learning to analyze and understand the customer's requirements are deployed. In addition, we use a face detection technique based on Multi-task Cascaded Convolutional Networks (MTCNN) to predict the distance between the robot and passengers to perform the function. The robot can guide passengers to desired areas in the terminal. The results and evaluation of the implementation process are also mentioned in the article and show promise.

Real-Time Emotional Analysis from A Live Webcam Using Deep Learning

In the Covid-19 age, we are becoming increasingly reliant on virtual interactions like as Zoom and Google meetings / Teams chat. The videos received from live webcamera in virtual interactions become great source for researchers to understand the human emotions. Due to the numerous applications in human-computer interaction, analysis of emotion from facial expressions has piqued the interest of the newest research community (HCI). The primary objective of this study is to assess various emotions using unique facial expressions captured via a live web camera. Traditional approaches (Conventional FER) rely on manual feature extraction before classifying the emotional state, whereas Deep Learning, Convolutional Neural Networks, and Transfer Learning are now widely used for emotional classification due to their advanced feature extraction mechanisms from images. In this implementation, we will use the most advanced deep learning models, MTCNN and VGG-16, to extract features and classify seven distinct emotions based on their facial landmarks in live video. Using the FER2013 standard dataset, we achieved a maximum accuracy of 97.23 percent for training and 60.2 percent for validation for emotion classification. © 2022 IEEE.

Masked Face Recognition and Liveness Detection Using Deep Learning Technique

Face recognition has been the most successful image processing application in recent times. Most work involving image analysis uses face recognition to automate attendance management systems. Face recognition is an identification process to verify and authenticate the person using their facial features. In this study, an intelligent attendance management system is built to automate the process of attendance. Here, while entering, a person’s image will get captured. The model will detect the face;then the liveness model will verify whether there is any spoofing attack, then the masked detection model will check whether the person has worn the mask or not. In the end, face recognition will extract the facial features. If the person’s features match the database, their attendance will be marked. In the face of the COVID-19 pandemic, wearing a face mask is mandatory for safety measures. The current face recognition system is not able to extract the features properly. The Multi-task Cascaded Convolutional Networks (MTCNN) model detects the face in the proposed method. Then a classification model based on the architecture of MobileNet V2 is used for liveness and mask detection. Then the FaceNet model is used for extracting the facial features. In this study, two different models for the recognition have been built, one for people with masks another one for people without masks. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Leveraging Discriminative Cues for Masked Face Recognition in Post COVID World

The post COVID world has completely disrupted our lifestyle, where wearing a mask is necessary to protect ourselves and others from contracting the virus. However, face masks have proved to be challenging for facial biometric systems, in the sense that these systems do not work as expected when wearing masks as nearly half of the face is covered, thus reducing discriminative features that the model can leverage. Most of the existing frameworks rely on the entire face as the input, but as the face is covered, these frameworks do not perform up to the mark. Moreover, training another facial recognition system with mask images is challenging as the availability of datasets is limited, both qualitatively and quantitatively. In this paper, we propose a framework that shows better results without significant training. In the proposed work, firstly we extracted the face using SSD, then by obtaining Facial Landmarks for utilizing the cues from other dis-criminative parts for facial recognition. The proposed framework is able to out-perform other frameworks on facial mask images and also found ~4.5% increment in accuracy. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Recognition of Student Emotions in an Online Education System

Online education system was developed due to the Covid-19 pandemic. The core idea of this paper is to map the connection between teaching practices to student learning in an online environment. Face to face evaluation techniques are fairly quick and easy for formative assessments to check student understanding in existent environment. Prevailing studies illustrate that a person's facial expressions and emotions are closely related. In order to make the teaching-learning process more effective, teachers usually collect day to day feedback from the students. This feedback can be used to improve teaching skills and make the process more interactive. In a virtual learning mode, there is a need to identify and understand the emotions of people. Constructive information can be extracted from online platforms using facial recognition algorithms. An online course connected with students is used for examination;the results have shown that this technique performs strongly. © 2021 IEEE.

Face Recognition with Mask Using MTCNN and FaceNet

Face recognition is a method of identifying or verifying the identity of an individual using their face but what if this recognition method could be extended further to suit the needs of the current scenario. Given this COVID pandemic, this paper fits best by recognizing the people wearing masks. The research has been done by creating our own dataset using images from our friends and relatives followed by doing image augmentation by performing operations like rotating by some angle, changing brightness and contrast, zooming in and out, etc. Then, face with the mask is extracted from the given image with the help of MTCNN to get a bounding box, width, and the height of the face, and then, segmentation has been done by reducing the height by a factor of 2. FaceNet pretrained model has been used to represent the faces on a 128-dimensional unit hyper-sphere and get the embeddings for further classification. Many different algorithms like linear Discriminant analysis, SVM, ridge classifier, K-neighbors classifier, logistic regression, Naive Bayes, XGBoost, Ada Boost, random forest classifier, and decision tree classifier have been used for experimentation. After testing this, good accuracy was obtained as can be seen in the result section of this paper. The scope of this paper is quite vast as it covers many practical applications in real-scenario like detecting the presence of a particular person from an image or even from video by capturing faces frame by frame. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.